Method and system for asymmetrical rate control for 3D video compression

ABSTRACT

A video transmitter compresses an uncompressed 3D video into a base view video and an enhancement view video using MPEG-4 MVC standard. The video transmitter allocates bits to compressed pictures of the uncompressed 3D video based on corresponding picture type. More bits are allocated to I-pictures than P-pictures, and more bits are allocated to P-pictures than B-pictures in a given coding view. More bits are allocated to a compressed picture of the base view video than a same type compressed picture of the enhancement view video. The correlation level between the base view video and the enhancement view video is utilized for bit-allocation in video compression. More bits are allocated to a picture in a lower coding layer than to the same type picture in a higher coding layer in a given coding view. Pictures with the same cording order are identified from different view videos for a joint bit-allocation.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

NOT APPLICABLE.

FIELD OF THE INVENTION

Certain embodiments of the invention relate to video processing. Morespecifically, certain embodiments of the invention relate to a methodand system for asymmetrical rate control for 3D video compression.

BACKGROUND OF THE INVENTION

Digital video capabilities may be incorporated into a wide range ofdevices such as, for example, cellular telephones, digital televisions,digital direct broadcast systems, digital video recording or capturedevices, and the like. Digital video devices may provide significantimprovements over conventional analog video systems in processing andtransmitting video sequences with increased bandwidth efficiency.

Video content may be recorded in two-dimensional (2D) format or inthree-dimensional (3D) format. In various applications such as, forexample, the DVD movies and the digital TV, a 3D video is oftendesirable because it is often more realistic to viewers than the 2Dcounterpart. A 3D video comprises a left view video and a right viewvideo. A 3D video frame may be produced by combining left view videocomponents and right view video components, respectively.

Various video encoding standards, for example, MPEG-1, MPEG-2, MPEG-4,H.263, and H.264/AVC, have been established for encoding digital videosequences in a compressed manner. A frame in a compressed video may becoded in three possible modes: I-picture, P-picture, and B-picture.Compressed video frames may be divided into groups of pictures (GOPs).For example, each GOP may comprise one I-picture, several P-picturesand/or several B-pictures.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of skill in the art, throughcomparison of such systems with some aspects of the present invention asset forth in the remainder of the present application with reference tothe drawings.

BRIEF SUMMARY OF THE INVENTION

A system and/or method is provided for asymmetrical rate control for 3Dvideo compression, substantially as shown in and/or described inconnection with at least one of the figures, as set forth morecompletely in the claims.

These and other features and advantages of the present invention may beappreciated from a review of the following detailed description of thepresent invention, along with the accompanying figures in which likereference numerals refer to like parts throughout.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary video coding system that isoperable to perform asymmetrical rate control for 3D video compression,in accordance with an embodiment of the invention.

FIG. 2 is a diagram illustrating an exemplary video transmission unitthat is operable to implement asymmetrical rate control for 3D videocompression, in accordance with an embodiment of the invention.

FIG. 3 is a diagram illustrating an exemplary independently decodable 3DAVC stream that is produced via asymmetrical rate control based onpicture type, in accordance with an embodiment of the invention.

FIG. 4 is a flow chart illustrating exemplary steps that are utilized toallocate bits based on picture type to produce an independentlydecodable 3D AVC stream, in accordance with an embodiment of theinvention.

FIG. 5 is a diagram illustrating an exemplary 3D layered compressedstream that is produced via asymmetrical rate control based on picturetype, in accordance with an embodiment of the invention.

FIG. 6 is a flow chart illustrating exemplary steps that are utilized toallocate bits based on picture type to produce a 3D layered compressedstream, in accordance with an embodiment of the invention.

FIG. 7 is a diagram illustrating an exemplary 3D layered compressedstream that is produced via jointly allocating bits to pictures in baseand enhancement view videos based on picture type, in accordance with anembodiment of the invention.

FIG. 8 is a flow chart illustrating exemplary steps that are utilized toperform a joint rate-control for a 3D layered compressed stream, inaccordance with an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Certain embodiments of the invention may be found in a method and/orsystem for asymmetrical rate control for 3D video. In variousembodiments of the invention, a video transmitter is operable to acquirean uncompressed three-dimensional (3D) video comprising a left viewvideo and a right view video. The video transmitter may be operable tocompress the acquired 3D video using MPEG-4 Multi-view Video Coding(MVC) standard to generate two coding views (a base view and anenhancement view). Each coding view comprises a plurality of layeredcompressed pictures for transmission. In this regard, the videotransmitter may be operable to allocate bits to each resultingcompressed picture of the acquired uncompressed 3D video based oncorresponding picture type such like I-picture, P-picture and B-picture.In a given coding layer, most bits available for each view video may beallocated to I-pictures, and more bits may be allocated to P-picturesthan B-pictures in the same view. More bits may be allocated to acompressed picture of the base view video than a same type compressedpicture of the enhancement view video. In the acquired uncompressed 3Dvideo, the correlation level between the base view video and theenhancement view video may be utilized for bit-allocation in videocompression. For a given coding view, more bits may be allocated to apicture in a lower coding layer than to the same type picture in ahigher coding layer. Target bit-rates may be determined and/orestablished for the associated base and enhancement view videos forvideo compression. The base view video and the enhancement view videomay be processed concurrently based on the corresponding determinedtarget bit-rates. Coding orders may be determined according to thedetermined target bit-rates. Pictures with the same coding order may beidentified for a joint bit-allocation. The identified compressedpictures with same coding order may be in different view videos.

FIG. 1 is a block diagram of an exemplary video coding system that isoperable to perform asymmetrical rate control for 3D video compression,in accordance with an embodiment of the invention. Referring to FIG. 1,there is shown a video transmission unit (VTU) 110, a communicationnetwork 120 and a video reception unit (VRU) 130.

The VTU 110 may comprise suitable logic, circuitry, interfaces and/orcode that may be operable to provide compressed video content to the VRU130. The VTU 110 may be operable to acquire an uncompressed 3D video andperform video compression on the acquired uncompressed 3D video. MPEG-4MVC standard may be applied to generate two coding views, namely a baseview and an enhancement view, and layered compressed pictures fortransmission. The resulting base and enhancement compressed videostreams may be multiplexed into a single stream (a transport stream)with a targeted bit rate for transmission. Various video compressionalgorithms such as, for example, run-length coding and/or Huffman orarithmetic codes, may be utilized to compress the acquired uncompressed3D video to a compressed picture sequence. The bit rate of eachcompressed picture may vary. In this regard, the VTU 110 may be operableto allocate variable number of bits to each compressed picture. The VTU110 may be operable to perform (bit) rate-control in video compressionbased on picture type. The rate-control may be performed using variousmethods, for example, a quantization level (QL) control method, toensure that a targeted bit rate is achieved and maintained in videocompression. Pictures in the compressed picture sequence may compriseI-pictures, P-pictures and/or B-pictures. Pictures in the compressedpicture sequence may be from the base view video, the enhancement viewvideo and may belong to different coding layers for a layered videocompression. In this regard, the rate-control may be managed in videocompression not only based on picture type like I-picture, P-picture andB-picture, but also based on associated coding view (base orenhancement) and/or coding layers. For example, more bits may beallocated to a picture in the base view video than to a same typepicture in the enhancement view video. In instances where a layeredvideo may be considered, layer 1, layer 2 and layer 3 may correspond tothe lowest coding layer (a base layer) to the highest coding layer invideo compression. More bits may be allocated to a picture in layer 1than to a same type picture in layer 2 and/or layer 3.

In a 3D video, the associated base view and enhancement view videosbelong to the same program. The correlation level between the base viewvideo and the enhancement view video may be considered for rate controlin video compression. For example, the VTU 110 may be operable to adjustor balance bit allocation between the base view video and theenhancement view video according to the corresponding correlation level.A joint rate control may be executed concurrently for both the base viewand the enhancement view of the 3D video. In this regard, pictures withthe same coding ordering may be a joint entity for bit-allocation. Acoding order is the order in which pictures may be compressed or codedby the VRU 130. The bit allocation may be determined and implementedjointly for pictures with the same coding ordering.

The communication network 120 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to provide platforms forcommunication between the VTU 110 and the VRU 130. The communicationnetwork 120 may be implemented as a wired or wireless communicationnetwork. The communication network 120 may be local area network, widearea network, the Internet, and the like.

The VRU 130 may comprise suitable logic, circuitry, interfaces and/orcode that may be operable to receive a transport stream from the VTU 110over the communication network 120. The received transport stream maycomprise coded or compressed 3D video streams of entertainment programssuch as, for example, a 3D TV program. The VRU 130 may be operable todecode the received compressed 3D video streams of the 3D TV programusing MPEG-4 Multi-view Video Coding (MVC) standard to generate a leftview and a right view. The generated left and right views may becomposed to present to users. Examples of the VRUs 130 through 150 maycomprise, for example, set-top boxes, personal computers, and the like.

In an exemplary operation, the VTU 110 may be operable to acquire anuncompressed 3D video comprising a left view video and a right viewvideo. The VTU 110 may be operable to use MPEG-4 MVC standard togenerate two coding views (a base view and an enhancement view) andlayered compressed pictures for transmission. The uncompressed 3D videomay be compressed picture-by-picture with a target bit-rate. The targetbit-rate may be controlled and maintained via adaptive bit-allocationbased on picture type and associated coding view and coding layers. TheVTU 110 may be operable to perform an adaptive bit-allocationpicture-by-picture such that the resulting compressed pictures may begenerated with corresponding target bit-rate. The bit-allocation may beadaptive to picture type like I-picture, P-picture and B-picture,associated coding view (base or enhancement) and/or coding layers. Thecorrelation level between the base view video and the enhancement viewvideo may be considered for rate control in video compression. Thegenerated 3D compressed video may be communicated with the VRU 130 viathe communication network 120. The VRU 130 may be operable to decode the3D compressed video from the VTU 110 using MPEG-4 MVC standard togenerate a left view and a right view for display. The generated leftand right views may be composed to present to users.

FIG. 2 is a diagram illustrating an exemplary video transmission unitthat is operable to implement asymmetrical rate control for 3D videocompression, in accordance with an embodiment of the invention.Referring to FIG. 2, there is shown a video transmission unit (VTU) 200.The VTU comprises a 3D video source 210, a base view encoder 212, anenhancement view encoder 214, FIFO buffers 216-218, a multiplexer 220, ahost processor 230 and a rate-control processor 240.

The 3D video source 210 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to capture uncompressed 3Dvideo contents. A 3D video comprises a left view video and a right viewvideo. A 3D video picture may be formed by combining left view videocomponents and right view video components. Each picture of the captureduncompressed 3D video is in a fixed size (in bits). In this regard, eachpicture of the captured uncompressed 3D video may be quantized to atarget bit-rate. The quantized pictures may be communicated withcorresponding video encoders such as the base view encoder 212 and theenhancement view encoder 214 for video compressing, for example, usingMPEG-4 MVC standard.

The base view encoder 214 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to encode the left viewvideo, for example, from the 3D video source 210 picture-by-picture. Thebase view encoder 214 may be operable to utilize various videocompression algorithms such as specified in MPEG-4, AVC, VC1, VP6,and/or other video formats to form compressed or coded video contentsfor the left view video from the 3D video source 210. Information suchas the scene information from base view coding may be communicated withthe enhancement view encoder 216 to be used for enhancement view codingby the enhancement view encoder 216. The base view encoder 214 may alsobe operable to receive information, comprising for example scene changeinformation from the enhancement view encoder 216.

The enhancement view encoder 216 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to encode the right viewvideo, for example, from the 3D video source 210 picture-by-picture. Theenhancement view encoder 216 may be operable to utilize various videocompression algorithms such as specified in MPEG-4, AVC, VC1, VP6,and/or other video formats to form compressed or coded video content forthe right view video from the 3D video source 210. The enhancement viewcoding may be correlated to the base view coding using the sceneinformation from the base view coding. Information such as the sceneinformation from enhancement view coding may be communicated to the baseview encoder 214 to be used for base view coding by the base viewencoder 214. The enhancement view encoder 216 may also be operable toreceive information comprising for example, scene change informationfrom the base view encoder 214.

The FIFO buffers 214-216 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to buffer or storecompressed pictures from the base view encoder 214 and the enhancementview encoder 216, respectively. The FIFO buffers 214-216 may operate ina first-in-first-out basis. The FIFO buffers 214-216 may be operable tomanage the buffered compressed pictures, which may be with correspondingtarget bit-rate, so as to be transmitted in an intended frame rate basedon, for example, QoS of targeted programs.

The multiplexer 230 may comprise suitable logic, circuitry, interfacesand/or code that may be operable to merge compressed video streams fromthe base view encoder 214 and the enhancement view encoder 216 into asingle video stream, namely a transport stream (TS), for transmission.

The host processor 230 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to control and manageassociated components such as, for example, the rate-control processor240, the base view encoder 214 and the enhancement view encoder 216. Thehost processor 230 may be operable to set up a target bit rate for videocompression to be maintained by the rate-control processor 240.

The rate-control processor 240 may comprise suitable logic, circuitry,interfaces and/or code that may be operable to allocate bits to eachpicture in video compression for the base view encoder 214 and theenhancement view encoder 216, respectively. The rate-control processor240 may be operable to allocate bits to each picture with a specificbit-rate according to corresponding picture information. Therate-control processor 240 may be operable to control and/or maintainthe rate-control based on picture type like I-picture, P-picture andB-picture, and associated coding view and/or coding layer information.For example, for pictures in a specific coding view of a coding layer,the rate control processor 240 may be operable to allocate most bits toan I-picture, more bits to a P-picture than a B-picture. A picture suchas a P-picture in the base view may be allocated more bits than a sametype picture, a P-picture, in the enhancement view. A picture in a lowercoding layer may be allocated more bits than a same type picture in ahigher coding layer. Since the base view video and the enhancement viewvideo are the two video streams of the same program, the correlationlevel of the base view video and the enhancement view video may beutilized for rate-control in video compression. For example, the ratecontrol processor 240 may be operable to allocate bits to pictures inthe base view video and in the enhancement view video, respectively,based on the corresponding correlation level between the base view videoand the enhancement view video.

Rate-control may be executed concurrently for pictures in both the baseview video and the enhancement view video. In video compression, withina GOP, subsequent pictures may be coded based on previous adjacentpictures. In this regard, the rate-control for pictures with the samecoding order may be processed jointly. The rate control processor 240may be operable to monitor the status of the FIFO buffers 214-216 and/orthe multiplexer 220 such that the rate-control may be managed ormaintained to support a specific transmission rate at the multiplexer220 without overflow and/or underflow in the FIFO buffers 214-216.

In an exemplary operation, the 3D video source 210 may be operable tocapture an uncompressed 3D video, which comprises a left view video anda right view video. The host processor 230 may be operable to set up acorresponding target bit rate for the base view encoder 214 and theenhancement view encoder 216, respectively. The rate-control processor240 may be operable to control and maintain the rate-control in videocompression with respect to the target bit-rate set by the hostprocessor 230. The rate-control processor 240 may be operable toallocate bits to pictures in different coding views and coding layersbased on picture type and associated coding view and coding layerinformation. The correlation level between the base view and theenhancement view video may be used for rate-control in videocompression. Pictures in the base view video and the enhancement viewvideo may be quantized according to the corresponding number ofallocated bits. The resulting quantized pictures in the base view videoand in the enhancement view video may be compressed or coded via thebase view encoder 212 and the enhancement view encoder 214,respectively. The resulting compressed pictures in the base view videoand the enhancement view video may be stored in the FIFO buffer 216 and218, respectively, for transmission. The multiplexer 220 may be operableto multiplex video streams from the FIFO buffer 216 and the FIFO buffer218 into a single transport stream to be transmitted to the VRU 130 viathe communication network 120.

FIG. 3 is a diagram illustrating an exemplary independently decodable 3DAVC stream that is produced via asymmetrical rate control based onpicture type, in accordance with an embodiment of the invention.Referring to FIG. 3, there is shown a 3D AVC stream 300. The 3D AVCstream 300 comprises a base view video stream 310 and an enhancementview video stream 320, which are generated or produced by the base viewencoder 212 and the enhancement view encoder 214, respectively, using,for example, MPEG-4 MVC standard. The base view video stream 310comprises a plurality of pictures, of which, an I-picture 301,P-pictures 303, 305, 307 and 309, and B-pictures 302, 304,306 and 308are illustrated. The enhancement view video stream 320 comprises aplurality of pictures, of which, P-pictures 311, 313, 315, 317 and 319,and B-pictures 312, 314, 416 and 318 are illustrated. The bit-rate foreach picture in the base view video stream 310 and the enhancement viewvideo stream 320 may be controlled and maintained by the rate-controlprocessor 240. A bit-rate of a picture in the 3D AVC stream 300 may bedetermined based on a picture type such as I-picture, P-picture andB-picture. For example, for a given coding view such as the base viewvideo stream 310, most bits available for the base view video stream 310may be allocated to I-pictures such as the I-picture 301, and more bitsmay be allocated to P-pictures than B-pictures in the enhancement viewvideo stream 310.

For a given amount of distortion, the base view video stream 310 and theright view video stream 320 may have different impacts on perceptualvisual quality of the 3D AVC stream 300. Accordingly, a picture in thebase view video stream 310 may be allocated more bits than the same typepicture in the enhancement view video stream 320. For example, more bitsmay be allocated to the B-picture 302 in the base view video stream 310than to the B-picture 312 in the enhancement view video stream 320.

FIG. 4 is a flow chart illustrating exemplary steps that are utilized toallocate bits based on picture type to produce an independentlydecodable 3D AVC stream, in accordance with an embodiment of theinvention. Referring to FIG. 4, the exemplary steps start with step 402,where parameter N_(bits) indicates the number of bits available forvideo compression in a coding view. Parameters N_I, N_P, and N_Bindicate number of bits available to be allocated to an I-picture, aP-picture and a B-picture, respectively, in the coding view for videocompression. In step 404, the 3D video source 210 may be operable tocapture an uncompressed 3D video and provide a picture of the captureduncompressed 3D video for video compression. In step 406, it may bedetermined whether the picture for video compression may be in the baseview video of the captured uncompressed 3D video. In instances where thepicture for video compression is in the base view video of the captureduncompressed 3D video, then in step 408, it may be determined whetherthe picture for video compression is an I-picture in video compression.In instances where the picture for video compression is an I-picture invideo compression, then in step 410, the picture may be allocated mostbits (N_I) out of the total bits available for the view. The exemplarysteps may be ended in step 418.

In step 408, in instances where the picture for video compression is notan I-picture, then in step 412, it may be determined whether the picturefor video compression is a P-picture. In instances where the picture forvideo compression is a P-picture, then in step 414, the picture may beallocated more bits (N_P) out of the total bits available for the viewand N_P<N_I. The exemplary steps may end at step 418.

In step 412, in instances where the picture for video compression is nota P-picture, then in step 416, where the picture may be allocated lessbits (N_B) out of the total bits available for the view and N_B<N_P. Theexemplary steps may end at step 418.

In step 406, in instances where the picture for video compression is notin the base view video of the captured uncompressed 3D video, then instep 420, where the number of bits available for the view, Nbits, may bereduced based on the correlation level between the base view video andthe enhancement view video of the received uncompressed 3D video. Theexemplary steps continue in step 408.

FIG. 5 is a diagram illustrating an exemplary 3D layered compressedstream that is produced via asymmetrical rate control based on picturetype, in accordance with an embodiment of the invention. Referring toFIG. 5, there is shown a 3D layered stream 500. The 3D layered stream500 comprises coding layers 510-530, which correspond to the base layer(layer 1), the middle layer (layer 2) and the high layer (layer 3),respectively, of the 3D layered stream 500. Two coding views aregenerated or produced in each coding layer using MPEG-4 MVC standard,for example. A base view video stream 510 a and an enhancement viewvideo stream 510 b are generated in the coding layer 510. Each videostream in the coding layer 510 may comprise a plurality of pictures, ofwhich pictures 511-518, 521-527, and 531-538 are illustrated. For agiven view, pictures in a higher coding layer may be coded based onadjacent pictures in lower coding layers. For example, in theenhancement view, the picture 536 in the coding layer 530 (layer 3) maybe coded based on the picture 516 in the coding layer 510 (layer 1) andthe picture 526 in the coding layer 520 (layer 2). As presented withrespect to FIG. 5, the coding layer 510 (layer 1) may be the startingcoding layer in the 3D layered stream 500, other coding layers such asthe coding layer 520 (layer 2) and the coding layer 530 (layer 3) may becoded depending on the coding layer 510 (layer 1). Accordingly,different importance of coding layers may exist in the 3D layered stream500. The coding layer 510 (layer 1) may be most important, and thecoding layer 520 (layer 2) may be more important than the coding layer530 (layer 3). In this regard, the number of bits, which may beallocated to each picture in the 3D layered stream 500, may bedetermined according to a descending order of importance from the codinglayer 510 (layer 1) to the coding layer (layer 3). For a given codingview such as a base view video, more bits may be allocated to a picturein a lower coding layer than to the same type picture in a higher codinglayer. For example, more bits may be allocated to the B-picture 521 inthe coding layer 520 than the pictures such as the B-pictures 531-532 inthe coding layer 530.

FIG. 6 is a flow chart illustrating exemplary steps that are utilized toallocate bits based on picture type to produce a 3D layered compressedstream, in accordance with an embodiment of the invention. Referring toFIG. 6, the exemplary steps start with step 602, where parameterN_(bits) indicates the number of bits available to be allocated picturesin a coding view for video compression. Parameters N_L1, N_L2, and N_L2indicate number of bits available to be allocated to pictures in layer1, layer 2 and layer 3, respectively, for video compression. In step604, the 3D video source 210 may be operable to capture an uncompressed3D video and provide a picture of the captured uncompressed 3D video forvideo compression. In step 606, it may be determined whether the picturemay be in the layer 1 for video compression. In instances where thepicture may be in the layer 1 for video compression, then in step 608,the picture may be allocated most bits (N_L1) out of the total bitsavailable for the coding view. The exemplary steps may end at step 616.

In step 606, in instances where the picture may not be in the layer 1for video compression, then in step 610, it may be determined whetherthe picture may be in the layer 2 for video compression. In instanceswhere the picture may be in the layer 2 for video compression, then instep 612, the picture may be allocated more bits (N_L2) out of the totalbits available for the coding view and N_L2<N_L1. The exemplary stepsmay end at step 616.

In step 610, in instances where the picture for video compression is notin the layer 2 for video compression, then in step 612, then in step614, the picture may be allocated less bits (N_L3) out of the total bitsavailable for the coding view and N_L3<N_L2. The exemplary steps may endat step 616.

FIG. 7 is a diagram illustrating an exemplary 3D layered compressedstream that is produced via jointly allocating bits to pictures in baseand enhancement view videos based on picture type, in accordance with anembodiment of the invention. Referring to FIG. 7, there is shown a 3Dlayered stream 700. The 3D layered stream 700 comprises coding layers710-730, which correspond to the base layer (layer 1), the middle layer(layer 2) and the high layer (layer 3), respectively, of the 3D layeredstream 700. Two coding views are generated in each coding layer usingMPEG-4 MVC standard. For example, a base view video stream 710 a and anenhancement view video stream 710 b are generated in the coding layer710. Each view stream comprises a plurality of pictures, of whichpictures 711-718, 721-727, and 731-738 in a GOP are illustrated. Forexample, the base view video stream 710 a comprises an I-picture 711,P-pictures 712 and 713. Pictures in the GOP may be coded based on theprevious adjacent pictures in the same coding view. For example, in acoding view, the picture 722 and the picture 733 are adjacent picturessubsequent to the picture 712 in the coding layer 720 and the codinglayer 730, respectively. Each of the picture 722 and the picture 733 maybe coded based on the picture 712. Pictures in the GOP are labeledaccording to corresponding coding order. For example, in instances wherethe picture 711 may be the start picture of the GOP, the picture 711 maybe coded first and the picture 712 may be coded second, and so on. Thecorresponding coding order number for the picture 711 and the picture712 are 1 and 2, respectively. In this regard, the rate-control forvideo compression may be performed on pictures according tocorresponding coding order numbers. Rate-control may be processedsimultaneously for pictures with the same coding order number. Forexample, the picture 717 and the picture 722 are with the same codingorder number of 7, a joint rate-control may be performed on the picture717 and the picture 722 simultaneously.

FIG. 8 is a flow chart illustrating exemplary steps that are utilized toperform a joint rate-control for a 3D layered compressed stream, inaccordance with an embodiment of the invention. Referring to FIG. 8, theexemplary steps start with step 802, where pictures of an uncompressed3D video may be labeled according to corresponding coding orders invideo compression. In step 804, pictures with the same coding ordernumber may be identified. In step 806, the rate-control may be performedor processed jointly on the identified pictures with the same codingorder number. The exemplary steps may end at step 808.

Aspects of a method and system for asymmetrical rate control for 3Dvideo are provided. In accordance with various embodiments of theinvention, a video transmitter such as the VTU 200 may be operable toreceive an uncompressed three-dimensional (3D) video via the 3D videosource 210. The received uncompressed 3D video comprises a left viewvideo and a right view video. The VTU 200 may be operable to compressthe received 3D video using MPEG-4 MVC standard to generate two codingviews (a base view and an enhancement view) and layered compressedpictures for transmission. In this regard, the VTU 200 may be operableto allocate bits to each resulting compressed picture of the receiveduncompressed 3D video based on corresponding picture type such likeI-picture, P-picture and B-picture. As described with respect to, forexample, FIG. 2-FIG. 5, in a given coding layer such as the coding layer510, most bits available for each view video such as the base view videostream 310, for example, may be allocated to I-pictures such as theI-picture 301, and more bits may be allocated to P-pictures thanB-pictures in the base view video stream 310. For example, in the baseview video stream 310, N_I bits may be allocated to an I-picture, N_Pbits to a P-picture, and N_B bits to a B-picture, respectively, whereN_I, N_P, and N_B are positive integers and N_I>N_P>N_B. More bits maybe allocated to a compressed picture of the base view video 310 than asame type compressed picture of the enhancement view video 320. Forexample, N_base bits may be allocated to the B-picture 302 in the baseview video stream 310 and N_enhancement bits may be allocated to theB-picture 312 in the enhancement view video stream 320, where N_base andN_enhancement are positive integers, and N_base>N_enhancement. In theacquired uncompressed 3D video, the associated base and enhancement viewvideos are the two video streams of the same program, the correlationlevel between the base view video and the enhancement view video may beutilized for bit-allocation in video compression. For example, the ratecontrol processor 240 may be operable to allocate bits to compressedpictures of the base view video and the enhancement view video,respectively, based on the corresponding correlation level between thebase view video and the enhancement view video.

In a 3D layered video stream such as the 3D layered video stream 500,compressed pictures in a high coding layer such as the coding layer 530may be derived from compressed pictures in low coding layers. For agiven coding view, more bits may be allocated to a picture in a lowercoding layer than to the same type picture in a higher coding layer. Forexample, N_low_layer bits may be allocated to the B-picture 521 in thecoding layer 520 and N_high_layer bits may be allocated to the picturessuch as the B-pictures 531-532 in the coding layer 530, N_low_layer andN_high_layer are positive integers, and N_low_layer>N_high_layer. For aspecific 3D program, the host processor 230 may be operable to determineor set up target bit-rates for the associated base view and enhancementview videos, respectively, for video compression. The host processor 230may be operable to control and manage, for example, the rate-controlprocessor 240, the base view encoder 212 and the enhancement viewencoder 214 such that the base view video and the enhancement view videomay be processed concurrently based on the corresponding determinedtarget bit-rates. Coding orders may be determined according to thedetermined target bit-rates. As described with respect to FIG. 2, FIG. 7and FIG. 8, the VTU 200 may be operable to perform compression coding onpictures according to the determined coding order. Pictures with thesame cording order may be identified for a parallel video compressioncoding. For example, the picture 717 and the picture 722 are with thesame coding order number of 7, a joint bit-allocation may be performedto the picture 717 and the picture 722 in video compression. Theidentified compressed pictures with same coding order may be indifferent view videos.

Another embodiment of the invention may provide a machine and/orcomputer readable storage and/or medium, having stored thereon, amachine code and/or a computer program having at least one code sectionexecutable by a machine and/or a computer, thereby causing the machineand/or computer to perform the steps as described herein for a methodand system for asymmetrical rate control for 3D video.

Accordingly, the present invention may be realized in hardware,software, or a combination thereof. The present invention may berealized in a centralized fashion in at least one computer system, or ina distributed fashion where different elements may be spread acrossseveral interconnected computer systems. Any kind of computer system orother apparatus adapted for carrying out the methods described hereinmay be suited. A typical combination of hardware and software may be ageneral-purpose computer system with a computer program that, when beingloaded and executed, may control the computer system such that itcarries out the methods described herein. The present invention may berealized in hardware that comprises a portion of an integrated circuitthat also performs other functions.

The present invention may also be embedded in a computer programproduct, which comprises all the features enabling the implementation ofthe methods described herein, and which when loaded in a computer systemis able to carry out these methods. Computer program in the presentcontext means any expression, in any language, code or notation, of aset of instructions intended to cause a system having an informationprocessing capability to perform a particular function either directlyor after either or both of the following: a) conversion to anotherlanguage, code or notation; b) reproduction in a different materialform.

While the present invention has been described with reference to certainembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted withoutdeparting from the scope of the present invention. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the present invention without departing from its scope.Therefore, it is intended that the present invention not be limited tothe particular embodiment disclosed, but that the present invention willinclude all embodiments falling within the scope of the appended claims.

What is claimed is:
 1. A method for video processing, the methodcomprising: performing by one or more processors and/or circuits in avideo processing system: compressing an uncompressed three-dimensional(3D) video having a first view video and a second view video into a baseview video and an enhancement view video, wherein the first view videois compressed into the base view video and the second view video iscompressed into the enhancement view video and wherein said compressingcomprises: allocating a first number of bits to a compressed picture ofthe base view video with a first picture type; and allocating a secondnumber of bits to a compressed picture of the enhancement view videowith the first picture type.
 2. The method according to claim 1,comprising for a given coding layer, allocating N_I bits to anI-picture, N_P bits to a P-picture, and N_B bits to a B-picture of saidbase view video and/or said enhancement view video, wherein N₁₃I, N_P,and N_B are positive integers and N₁₃ I>N_P >N_B.
 3. The methodaccording to claim 2, comprising for said given coding layer, allocatingN_base bits to a compressed picture of said base view video andN_enhancement bits to a compressed picture of said enhancement viewvideo, wherein said compressed picture of said base view video is ofsame picture type as said compressed picture of said enhancement viewvideo same type, wherein N_base and N_enhancement are positive integers,and N_base >N enhancement.
 4. The method according to claim 2,comprising for said given coding layer, allocating bits to eachcompressed picture of said base view video and/or said enhancement viewvideo based on a correlation level between said base view video and saidenhancement view video.
 5. The method according to claim 1, comprisingfor a given view video, allocating N_low_layer bits to a compressedpicture in a low coding layer and N_high_layer bits to a compressedpicture in a high coding layer, wherein said compressed picture in saidlow coding layer is of same picture type as said compressed picture insaid high coding layer, wherein N_low_layer and N_high_layer arepositive integers, and N_low_layer >N_high_layer.
 6. The methodaccording to claim 1, comprising determining target bit-rates for saidbase view video and said enhancement view video, respectively.
 7. Themethod according to claim 6, comprising determining coding orders ofpictures for said base view video and said enhancement view videoaccording to said determined target bit-rates.
 8. The method accordingto claim 7, comprising identifying compressed pictures with same codingorder according to said determination.
 9. The method according to claim8, comprising jointly allocating bits to said identified compressedpictures.
 10. The method according to claim 8, wherein said identifiedcompressed pictures with same coding order are in different view videos.11. A system for video processing, the system comprising: one or moreprocessors and/or circuits for use in a video processing system, whereinsaid one or more processors and/or circuits are operable to compress anuncompressed three-dimensional (3D) video into a base view video and anenhancement view video, wherein said compressing comprises allocatingbits between a first compressed picture in the base view video and asecond compressed picture in the enhancement view video based oncorresponding picture type, coding layer and correlation level betweenthe first compressed picture in the base view video and the secondcompressed picture in the enhancement view video.
 12. The systemaccording to claim 11, wherein said one or more processors and/orcircuits are operable to, for a given coding layer, allocate N_I bits toan I-picture, N_P bits to a P-picture, and N_B bits to a B-picture ofsaid base view video and/or said enhancement view video, wherein N_I,N_P, and N_B are positive integers and N_I>N_P>N_B.
 13. The systemaccording to claim 12, wherein said one or more processors and/orcircuits are operable to, for said given coding layer, allocate N_basebits to a compressed picture of said base view video and N_enhancementbits to a compressed picture of said enhancement view video, whereinsaid compressed picture of said base view video is of same picture typeas said compressed picture of said enhancement view video same type,wherein N_base and N_enhancement are positive integers, andN_base >N_enhancement.
 14. The system according to claim 12, whereinsaid one or more processors and/or circuits are operable to, for saidgiven coding layer, allocate bits to each compressed picture of saidbase view video and/or said enhancement view video based on acorrelation level between said base view video and said enhancement viewvideo.
 15. The system according to claim 11, wherein said one or moreprocessors and/or circuits are operable to, for a given view video,allocate N_low_layer bits to a compressed picture in a low coding layerand N_high_layer bits to a compressed picture in a high coding layer,wherein said compressed picture in said low coding layer is of samepicture type as said compressed picture in said high coding layer,wherein N_low_layer and N_high_layer are positive integers, andN_low_layer >N_high_layer.
 16. The system according to claim 11, whereinsaid one or more processors and/or circuits are operable to determinetarget bit-rates for said base view video and said enhancement viewvideo, respectively.
 17. The system according to claim 16, wherein saidone or more processors and/or circuits are operable to determine codingorders of pictures for said base view video and said enhancement viewvideo according to said determined target bit-rates.
 18. The systemaccording to claim 17, wherein said one or more processors and/orcircuits are operable to identify compressed pictures with same codingorder according to said determination.
 19. The system according to claim18, wherein said one or more processors and/or circuits are operable tojointly allocate bits to said identified compressed pictures.
 20. Thesystem according to claim 18, wherein said identified compressedpictures with same coding order are in different view videos.